Network-based ultrasound imaging system

ABSTRACT

Systems and methods for network-based ultrasound imaging are provided, which can include a number of features. In some embodiments, an ultrasound imaging system images an object with three-dimensional unfocused pings and obtains digital sample sets from a plurality of receiver elements. A sub-set of the digital sample sets can be electronically transferred to a remote server, where the sub-set can be beamformed to produce a series of two-dimensional image frames. A video stream made up of the series of two-dimensional images frames can then be transferred from the remote server to a display device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/500,933, filed Feb. 1, 2017, now U.S. Pat. No. 10,401,493, whichapplication is the national stage under 35 USC 371 of InternationalApplication No. PCT/US2015/045703, filed Aug. 18, 2015, whichapplication claims the benefit of U.S. Provisional Patent ApplicationNo. 62/038,602, filed Aug. 18, 2014, titled “Network-Based UltrasoundImaging System”, the contents of which are incorporated by referenceherein.

INCORPORATION BY REFERENCE

All publications and patent applications mentioned in this specificationare herein incorporated by reference to the same extent as if eachindividual publication or patent application was specifically andindividually indicated to be incorporated by reference.

FIELD

This application relates generally to the field of ultrasound imaging,and more particularly to a network-based ultrasound imaging system.

BACKGROUND

Ultrasound imaging provides for relatively low cost medical andnon-medical imaging without the risks associated with ionizingradiation, or the complications of MRI imaging. Improvements toultrasound imaging techniques combined with improvements toclient-server networking architecture may provide additionalopportunities for the use of ultrasound imaging to solve imagingchallenges.

SUMMARY OF THE DISCLOSURE

A method of ultrasound imaging is provided, comprising transmitting anunfocused three-dimensional ping into an object from a transmitterelement of a transducer array in a probe of a data capture device,receiving echoes of the unfocused three-dimensional ping with aplurality of receiver elements of the transducer array, convertinganalog signals from each of the plurality of receiver elements into afull dataset of digital sample sets, wherein the full dataset comprisesdigital sample sets from all the receiver elements, electronicallytransferring a sub-set of the digital sample sets to a remote server,wherein the sub-set comprises fewer digital samples than the fulldataset, beamforming the sub-set of digital samples in the remote serverto produce a series of two-dimensional image frames, and transferring avideo stream made up of the series of two-dimensional image frames fromthe remote server to a display device.

In some embodiments, the method further comprises, in response to acontrol signal, transferring the full dataset from the data capturedevice to the remote server and storing the full dataset at the remoteserver.

In another embodiment, the method further comprises determining digitalsamples to include in the sub-set of digital samples by identifyingdigital samples associated with a selected imaging window from among thefull dataset of digital sample sets.

In some embodiments, the display device is physically attached to thedata capture device. In other embodiments, the display device is notphysically attached to the data capture device. In further embodiments,the display device is a mobile device.

In some embodiments, the method further comprises selecting digitalsamples to include in the sub-set of digital samples by selecting onlydata samples corresponding to less than all pings transmitted from theprobe.

In one embodiment, the method further comprises selecting digitalsamples to include in the sub-set of digital samples by selecting onlydata samples corresponding to less than all receive elements of thearray.

In other embodiments, the method further comprises selecting digitalsamples to include in the sub-set of digital samples by selecting onlydata samples corresponding to less than all receive apertures of thearray.

A network-based imaging system is provided, comprising a data capturedevice comprising a housing containing transmit control electronicsconfigured to transmit ultrasound signals from a first plurality oftransducer elements, receiver electronics configured to receive echoesof the transmitted ultrasound signals, the receiver electronics beingfurther configured to digitize and store the received echoes as a fulldataset in a first memory device physically located within a commonhousing of the data capture device, and communication electronicsconfigured to communicate the full dataset. The system further comprisesa remote server device comprising server communication electronicsconfigured to receive the digitized echoes communicated by thecommunication electronics of the data capture device, beamformingsoftware executed by the remote server device and configured to convertthe received digitized echoes into a video stream of consecutive imageframes, video streaming software executed by the remote server deviceand configured to stream the video to a display device.

In some embodiments, the display device further comprises user interfacesoftware executed by the display device and configured to receive userinputs to control one or more beamforming or video streaming parameters,and further configured to transfer user inputs to the beamformingsoftware at the remote server device, the user interface softwarefurther comprising a user input control configured to transfer the fulldataset to the remote server, and video display software executed by thedisplay device and configured to receive the video stream from theremote server device and to display the video stream.

In some embodiments, the system further comprises a plurality of datacapture devices in communication with the remote server device.

A method of collecting volumetric data representing a target object isprovided, the method comprising transmitting an unfocusedthree-dimensional ping into the target object from a transmitter elementof a transducer array in a probe, receiving echoes of the unfocusedthree-dimensional ping with a plurality of receiver elements of thetransducer array, converting analog signals from each of the pluralityof receiver elements into a full dataset of digital sample sets, whereinthe full dataset comprises digital sample sets from all the receiverelements, electronically transferring a sub-set of the digital samplesets to a remote server, wherein the sub-set comprises fewer digitalsamples than the full dataset, beamforming the sub-set of digitalsamples in the remote server to produce a series of two-dimensionalimage frames, transferring a video stream made up of the series oftwo-dimensional image frames from the remote server to a mobile displaydevice.

In some embodiments, the method further comprises, in response to acontrol signal, transferring the full dataset to the remote server andstoring the full dataset at the remote server.

A method of ultrasound imaging is also provided comprising transmittinga plurality of unfocused three-dimensional pings into athree-dimensional target volume from a plurality of transmitter elementsof a transducer array in a probe, receiving echoes of the unfocusedthree-dimensional pings with a plurality of receiver elements of thetransducer array, converting analog signals from each of the pluralityof receiver elements into a full dataset of digital sample sets, whereinthe full dataset comprises digital sample sets from all the receiverelements, selecting a two-dimensional plane intersecting thethree-dimensional target volume, identifying three-dimensional voxelsintersecting the selected two-dimensional plane, identifying a sub-setof data samples corresponding to the selected two-dimensional plane,communicating only the sub-set of data samples over a computer networkto a remote server, receiving, from the remote server, a video stream oftwo-dimensional images representing the selected two-dimensional plane,and displaying the video stream on a display device adjacent to theprobe.

In some embodiments, the method further comprises, in response to a usercommand, communicating the full dataset to a remote data storage device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a process flow diagram illustrating an example process fordirectly beamforming a two-dimensional image plane from raw echo dataobtained from a three-dimensional volume.

FIG. 2 is a schematic perspective illustration of a multiple apertureimaging system for imaging a three-dimensional volume.

FIG. 3 is a schematic perspective illustration of a multiple apertureimaging system imaging a two-dimensional plane within athree-dimensional volume.

FIG. 4 is a schematic illustration of an embodiment of an imaging systemincorporating a raw data memory.

FIG. 5 is a schematic illustration of an embodiment of a network-basedimaging system.

FIG. 6 is a schematic illustration of another embodiment of anetwork-based imaging system.

FIG. 7 is a process flow diagram illustrating an example embodimentnetwork-based imaging process.

FIG. 8 is a schematic illustration of another embodiment of anetwork-based imaging system.

FIG. 9 is a process flow diagram illustrating an example embodimentnetwork-based imaging process.

DETAILED DESCRIPTION Introduction and Definitions

Although the various embodiments are described herein with reference toultrasound imaging of various anatomic structures, it will be understoodthat many of the methods and devices shown and described herein may alsobe used in other applications, such as imaging and evaluatingnon-anatomic structures and objects. For example, the probes, systemsand methods described herein may be used in non-destructive testing,inspection or evaluation of various mechanical objects, structuralobjects or materials, such as welds, pipes, beams, plates, pressurevessels, layered structures, etc. The various embodiments below includesystems and methods for using an ultrasound imaging system that isconfigured to store raw, un-beamformed ultrasound data for subsequentbeamforming and processing into image data. Such a system enables manyunique methods of using ultrasound imaging systems.

Although examples are described herein with reference to transmission ofultrasound impulses into a medium to be imaged and reception of echoesof the transmitted ultrasound impulses. However, the skilled artisanwill recognize that many of the techniques and systems described hereinmay be equally applicable to transmission and reception of other formsof energy, such as electromagnetic radiation including radio frequencysignals, microwave signals, X-rays, or any other part of theelectromagnetic spectrum.

As used herein the terms “ultrasound transducer” and “transducer” maycarry their ordinary meanings as understood by those skilled in the artof ultrasound imaging technologies, and may refer without limitation toany single component capable of converting an electrical signal into anultrasonic signal and/or vice versa. For example, in some embodiments,an ultrasound transducer may comprise a piezoelectric device. In someother embodiments, ultrasound transducers may comprise capacitivemicromachined ultrasound transducers (CMUT) or any otherelectro-acoustic transducer device. In some embodiments, transducers maycomprise components for the transduction of other energy forms, such aselectromagnetic radiation.

Transducers are often configured in arrays of multiple individualtransducer elements. As used herein, the terms “transducer array” or“array” generally refers to a collection of transducer elements mountedto a common backing block. Such arrays may have one dimension (1D), twodimensions (2D), 1.X dimensions (1.XD) or three dimensions (3D). Otherdimensioned arrays as understood by those skilled in the art may also beused. Annular arrays, such as concentric circular arrays and ellipticalarrays may also be used. An element of a transducer array may be thesmallest discretely functional component of an array. For example, inthe case of an array of piezoelectric transducer elements, each elementmay be a single piezoelectric crystal or a single machined section of apiezoelectric crystal.

As used herein, the terms “transmit element” and “receive element” maycarry their ordinary meanings as understood by those skilled in the artof ultrasound imaging technologies. The term “transmit element” mayrefer without limitation to an ultrasound transducer element which atleast momentarily performs a transmit function in which an electricalsignal is converted into an ultrasound signal. Similarly, the term“receive element” may refer without limitation to an ultrasoundtransducer element which at least momentarily performs a receivefunction in which an ultrasound signal impinging on the element isconverted into an electrical signal. Transmission of ultrasound into amedium may also be referred to herein as “insonifying.” An object orstructure which reflects ultrasound waves may be referred to as a“reflector” or a “scatterer.”

As used herein, the term “aperture” may refer to a conceptual “opening”through which ultrasound signals may be sent and/or received. In actualpractice, an aperture is simply a single transducer element or a groupof transducer elements that are collectively managed as a common groupby imaging control electronics or by beamforming electronics orsoftware. For example, in some embodiments an aperture may be a physicalgrouping of elements which may be physically separated from elements ofan adjacent aperture. However, adjacent apertures need not necessarilybe physically separated.

It should be noted that the terms “receive aperture,” “insonifyingaperture,” and/or “transmit aperture” are used herein to mean anindividual element, a group of elements within an array, or even entirearrays within a common housing, or groups of elements in multipleseparate arrays, that perform the desired transmit or receive functionfrom a desired physical viewpoint or aperture. In some embodiments, suchtransmit and receive apertures may be created as physically separatecomponents with dedicated functionality. In other embodiments, anynumber of send and/or receive apertures may be dynamically definedelectronically as needed. In other embodiments, a multiple apertureultrasound imaging system may use a combination of dedicated-functionand dynamic-function apertures.

As used herein, the term “total aperture” refers to the total cumulativesize of all imaging apertures. In other words, the term “total aperture”may refer to one or more dimensions defined by a maximum distancebetween the furthest-most transducer elements of any combination of sendand/or receive elements used for a particular imaging cycle. Thus, thetotal aperture is made up of any number of sub-apertures designated assend or receive apertures for a particular cycle. In the case of asingle-aperture imaging arrangement, the total aperture, sub-aperture,transmit aperture, and receive aperture will all have the samedimensions. In the case of a multiple array probe, the dimensions of thetotal aperture may include the sum of the dimensions of all of thearrays.

As used herein, the term “ping cycle” may refer to a cycle beginningwith a ping signal being transmitted from a transmit aperture and echoesof that ping being received by receiver transducer elements. In somecases, echoes from two or more pings may be combined to form a singleimage frame, and multiple frames may be displayed in sequence to form avideo. Thus, an “image cycle” may contain echoes from multiple pingcycles. In other cases, a single ping cycle may correspond to a singleimage cycle.

In some embodiments, two apertures may be located adjacent one anotheron a continuous array. In still other embodiments, two apertures mayoverlap one another on a continuous array, such that at least oneelement functions as part of two separate apertures. The location,function, number of elements and physical size of an aperture may bedefined dynamically in any manner needed for a particular application.Constraints on these parameters for a particular application will bediscussed below and/or will be clear to the skilled artisan.

Elements and arrays described herein may also be multi-function. Thatis, the designation of transducer elements or arrays as transmitters inone instance does not preclude their immediate redesignation asreceivers in the next instance. Moreover, embodiments of the controlsystem herein include the capabilities for making such designationselectronically based on user inputs, pre-set scan or resolutioncriteria, or other automatically determined criteria.

As used herein the term “point source transmission” or “ping” may referto an introduction of transmitted ultrasound energy into a medium from asingle spatial location. This may be accomplished using a singleultrasound transducer element or combination of adjacent transducerelements transmitting together as a single transmit aperture. A singletransmission from a point source transmit aperture approximates auniform spherical wave front, or in the case of imaging a 2D slice, auniform circular wave front within the 2D slice. In some cases, a singletransmission of a circular or spherical wave front from a point sourcetransmit aperture may be referred to herein as a “ping” or a “pointsource pulse.”

As used herein, the phrase “pixel resolution” refers to a measure of anumber of pixels in an image, and may be expressed with two positiveintegers, the first referring to a number of pixel columns (image width)and the second referring to a number of pixel rows (image height).Alternatively, pixel resolution may be expressed in terms of a totalnumber of pixels (e.g., the product of the number of rows and the numberof columns), a number of pixels per unit length, or a number of pixelsper unit area. “Pixel resolution” as used herein is distinct from otheruses of the term “resolution” which refers to the level of detailvisible in an image. For example, “lateral resolution” may refer to thelevel of detail that may be discerned along a horizontal axis in anultrasound image plane, independent of how an image of such a plane maybe represented as a digital image made up of pixels.

Ping-Based Ultrasound Imaging

In various embodiments, point-source transmission ultrasound imaging,otherwise referred to as ping-based ultrasound imaging, provides severaladvantages over traditional scanline-based imaging. Point sourcetransmission differs in its spatial characteristics from a “phased arraytransmission” which focuses energy in a particular direction from thetransducer element array along a directed scanline. An unfocused pointsource pulse (ping) may be transmitted so as to insonify as wide an areaas possible with an unfocused wavefront.

In some cases, an unfocused “circular” wavefront may be transmitted intoa single image plane or “scanning plane.” Such two-dimensional focusingmay be achieved by providing a lensing material between a transducerelement and an object to be imaged. A ping focused into a single planemay be referred to as a two-dimensional ping.

For volumetric imaging, an unfocused three-dimensional ping may betransmitted to form a substantially spherical wavefront which may bereferred to as a three-dimensional ping.

Echoes of a transmitted ping will be returned from scatterers in theregion of interest and may be received by all of the receiver elementsof a probe. The receiver elements may be grouped into “receiveapertures” as will be further described below. Those echo signals may befiltered, amplified, digitized and stored in short term or long termmemory (depending on the needs or capabilities of a particular system).

Images may then be reconstructed from received echoes by assuming thatthe wavefronts emitted from the point source are physically circular(for 2D imaging) or spherical (for 3D imaging) in the region ofinterest. In actuality, a two-dimensionally focused wavefront may alsohave some penetration in the dimension normal to the 2D image plane.That is, some energy may essentially “leak” into the dimensionperpendicular to the desired two-dimensional imaging plane.

Additionally, a “circular” wavefront may be limited to a semicircle or afraction of a circle less than 180 degrees ahead of the front face ofthe transducer according to the unique off-axis properties of atransducing material. Similarly, when transmitting three-dimensional“spherical” pings, the corresponding wavefronts may have a shape of asemi-sphere or a smaller fractional sphere section (e.g., a “cone”shape), depending on the off-axis characteristics of the transmitelement(s) used.

The process of forming an image from received echoes is generallyreferred to herein as “beamforming.” In ping-based imaging, beamformingmay generally involve determining an echo sample corresponding to eachpixel or voxel position within an image window. Alternately, beamformingmay involve the reverse, that is determining a pixel display locationfor each received echo sample. Because each ping insonifies an entireimaged region, a “complete” (albeit low quality) image may be formedwith the echoes of a single transducer element. An image that may beformed from echoes received by a single receive transducer element maybe referred to as a sub-image. The image quality may be improved bycombining sub-images formed from echoes received at a plurality oftransducer elements. Transducer elements may be grouped into“apertures,” and sub-images from elements of a common aperture may becombined to form an image layer.

Beamforming of ping-based echoes may be performed using a software-basedor hardware-based dynamic beamforming technique, in which a beamformer'sfocus may be continuously changed to focus at a particular pixelposition as that pixel is being imaged. Such a beamformer may be used toplot the position of echoes received from a point source pulse. In someembodiments, a dynamic beamformer may plot the locus of each echo signalbased on a round-trip travel time of the signal from the transmitter toan individual receive transducer element.

When beamforming echoes of a transmitted two-dimensionally focused ping,the locus of a single reflector will lie along an ellipse with a firstelliptical focus at the position of the transmit transducer element(s)and the second elliptical focus at the position of the receivetransducer element. Although several other possible reflectors lie alongthe same ellipse, echoes of the same reflector will also be received byeach of the other receive transducer elements of a receive aperture. Theslightly different positions of each receive transducer element meansthat each receive element will define a slightly different ellipse for agiven reflector. Accumulating the results by coherently summing theellipses for all elements of a common receive aperture will indicate anintersection of the ellipses for the reflector, thereby convergingtowards a point at which to display a pixel representing the reflector.The echo amplitudes received by any number of receive elements maythereby be combined into each pixel value. In other embodiments thecomputation can be organized differently to arrive at substantially thesame image.

When beamforming echoes of a transmitted three-dimensional ping,substantially the same process may be followed, but the possible locusof each reflector lies on a three-dimensional ellipsoid with a firstellipsoidal focus at the position of the transmit transducer element,and a second ellipsoidal focus at the position of the receivingtransducer element. Therefore, combining impressions of a particularreflector obtained with multiple receive elements may produce a voxelpoint at the three-dimensional intersection of the multiple ellipsoids.

Errors in information describing the relative three-dimensional positionof transmitting and receiving elements may substantially degrade imagequality. Therefore, a calibration process may be used to minimize errorin element position information.

Various algorithms may be used for combining echo signals received byseparate receive elements. For example, some embodiments may processecho signals individually, plotting each echo signal at all possiblelocations along its ellipse, then proceeding to the next echo signal.Alternatively, each pixel location may be processed individually,identifying and processing all echoes potentially contributing to thatpixel location before proceeding to the next 2D pixel or 3D voxellocation.

Image quality may be further improved by combining images formed by thebeamformer from one or more subsequent transmitted pings, transmittedfrom the same or a different point source (or multiple different pointsources). Improvements to image quality may be obtained by combiningimages formed by more than one receive aperture. The process ofcombining separately beamformed images, pixels or voxels may be referredto herein as “image layer combining.” Combining images from echoesreceived at multiple, separate apertures of a multiple apertureultrasound probe may further improve image quality. The term “imagelayer combining” may refer to the combination of two or more overlappingpixel values, voxel values, or complete images (i.e., arrays of pixeland/or voxel values), where the overlapping values are obtained usingdifferent transmitted pings, different transmit apertures, and/ordifferent receive apertures. Examples of image layer combining processesare described in Applicant's prior applications referenced andincorporated by reference herein.

In some embodiments, ping-based multiple aperture imaging may operate bytransmitting a point-source ping (e.g., a 2D ping or a 3D ping) from afirst transmit aperture and receiving echoes with elements of two ormore receive apertures, one or more of which may include some or allelements of a transmit aperture. An image may be formed by triangulatingthe position of scatterers based on delay times between pingtransmission and reception of echoes, the speed of sound, and therelative positions of transmit and receive transducer elements. As aresult, a sub-image of the entire insonified region may be formed fromechoes of each transmitted ping received by each receive element.Coherently combining sub-images from echoes received by multipleelements grouped into a first receive aperture may produce theimprovement described above with reference to intersecting ellipses.Sub-images from echoes received by multiple elements grouped into asecond receive aperture may also be coherently combined with oneanother, and then the first-aperture image and the second-aperture imagemay be combined coherently or incoherently.

In some embodiments, a single time domain frame may be formed bycombining images formed from echoes received at two or more receiveapertures from a single transmitted ping. In other embodiments, a singletime domain frame may be formed by combining images formed from echoesreceived at one or more receive apertures from two or more transmittedpings. In some such embodiments, the multiple transmitted pings mayoriginate from different transmit apertures.

The same ping-based imaging techniques may be applied to 3D volumetricdata by transmitting ping signals that are not constrained to a singleplane (e.g., three-dimensional semi-spherical or near-semi-sphericalultrasound signals), and receiving echoes with receive elementsdisplaced from one another in two dimensions perpendicular a lineextending into the imaged medium, as described herein and in Applicant'sprevious applications. Multiple aperture ultrasound probes configuredfor ping-based 3D volumetric imaging may have large total apertures,which may be substantially greater than any expected coherence width foran intended imaging application. Examples of multiple apertureultrasound probes are shown and described in Applicant's U.S. patentapplication Ser. No. 13/272,105, published as US 2012/0095343 (now U.S.Pat. No. 9,247,926), and U.S. patent application Ser. No. 14/279,052(now U.S. Pat. No. 9,883,848), both of which are incorporated byreference herein.

3D volumetric data may be captured and stored using substantially thesame systems and methods described above. Typically, a multiple apertureprobe for 3D imaging may have substantially more transducer elementsthan a probe intended primarily for 2D imaging. As such, an imagingsystem for capturing and storing 3D volumetric data during a ping-basedimaging process may include substantially more receive channels and mayalso include a larger capacity raw data memory device. The raw echo dataobtained with a probe for 3D volumetric imaging may be stored in thememory device. Such volumetric raw data may be structured similarly todata captured with a probe configured for 2D imaging, such that echoesmay be distinguished based on the particular receive element thatreceived them and the particular transmitted ping that generated theechoes.

Beamforming 3D ping-based echo data may also be performed using similarsystems and methods to those used for beamforming 2D ping-based echodata. Each digitized sample value may represent a scatterer from theinsonified region of interest. As in the 2D case, the amplitude of eachreceived sample along with its time of arrival and the exactthree-dimensional positions of the transmitting and receivingtransducers may be analyzed to define a locus of points identifyingpotential positions of the scatterer. In the 3D case, such a locus is athree-dimensional ellipsoid having as its foci the positions of thetransmitting and receiving transducer elements. Each unique combinationof transmitting and receiving transducer elements may define a separateview of the same reflector. Thus, by combining information from multipletransmit-receive transducer element combinations, the actualthree-dimensional location of each reflector may be more accuratelyrepresented as a three-dimensional point or voxel in a three-dimensionalvolume.

For example, in some embodiments a 3D array of voxels representingreflectors in a 3D volume may be assembled in computer memory bybeginning with an evaluation of a selected digital sample. The selecteddigitized sample value may be written into every voxel indicated by thecorresponding ellipsoid as described above. Proceeding to do the samewith every other collected sample value, and then combining allresulting ellipsoids may produce a more refined image. Real scatterersmay be indicated by the intersection of many ellipsoids whereas parts ofthe ellipsoids not reinforced by other ellipsoids may have low levels ofsignal and may be eliminated or reduced by filters or other imageprocessing steps.

In other embodiments, the order of computation may be changed bybeginning with a selected voxel in a final 3D volume representation tobe produced. For example, for a selected voxel, the closest storedsample may be identified for each transmitter/receiver pair. All samplescorresponding to the selected voxel (i.e., all samples with an ellipsoidthat intersects the voxel) may then be evaluated and summed (oraveraged) to produce a final representation of the voxel. Closeness of asample to a selected voxel may be determined by calculating the vectordistance from the three-dimensional position of a transmitter (i.e., thetransmitter from which the ping signal was transmitted to produce theecho sample) to the selected voxel position plus the vector distancefrom the selected voxel position to the position of a receiver at whichthe sample was received. Such a linear distance may be related to thetime-divided sample values by dividing the total path length by speed ofsound through the imaged object. If received data samples are storedand/or indexed based on the time after a transmitted ping at which theywere received, then samples corresponding to a particular voxel may beidentified based on the element position data and speed-of-sound data asdescribed above. Using such methods, the samples corresponding to acalculated time may be associated with the selected voxel.

In some embodiments, a voxel of a final 3D volume representation may bemade up of combined data from multiple receive elements, from multiplereceive apertures, from multiple pings, or various combinations ofthese. An example will now be described with reference to an arbitrarilyselected voxel. A first ping signal may be transmitted from a firsttransmit element, and the echoes received by each receive elements maybe digitized and stored separately (e.g., one echo string per receiveelement per ping). A first set of echo data may be identified asrepresenting energy from the first ping corresponding to the selectedvoxel received by elements of a first receive aperture. A second set ofecho data generated from the first ping may also be identified ascorresponding to the selected voxel received by elements of a secondreceive aperture.

A second ping signal may then be transmitted from a second, differenttransmit element. A third set of echo data representing energy from thesecond ping may be identified as corresponding to the selected voxelreceived by elements of the first receive aperture. A fourth set of echodata of the second ping may be identified as corresponding to theselected voxel received by elements of the second receive aperture.

As will be clear in view of the present disclosure, data received eachelement of the first receive aperture may provide a separaterepresentation of each voxel in the imaged volume. Thus, the first dataset may contain multiple data points representing the selected voxel asreceived by the individual elements of the first receive aperture. Thedata points of the first data set may be coherently combined with oneanother to produce a first impression of the selected voxel. The datapoints of the second data set representing signals from the first pingreceived by elements of the second receive aperture may be coherentlycombined with one another to produce a second impression of the selectedvoxel. The data points of the third data set representing signals fromthe second ping received by elements of the first receive aperture maybe coherently combined with one another to produce a third impression ofthe example. The data points of the fourth data set representing signalsfrom the second ping received by elements of the second receive aperturemay be coherently combined with one another to produce a fourthimpression of the selected voxel.

The first selected voxel impression may be coherently combined with thethird selected voxel impression to form a first combined voxelimpression of the selected voxel. Because both the first impression andthe third impression were obtained with the same receive aperture, theymay be combined coherently without risking phase cancellation (assumingthe first receive aperture is sized to be less than a maximum coherentwidth for an intended imaging application).

The second impression of the selected voxel may be coherently combinedwith the fourth impression to form a second combined voxel impression ofthe selected voxel.

In some embodiments the step of coherently combining data from the firstping received by the elements of the first aperture with the data fromthe second ping received by the same elements of the same first receiveaperture may be performed before any other combining steps. In someembodiments, combination of signals from two separate pings received bythe same receive elements may be performed before or simultaneously withcombining signals received by elements of a common receive aperture. Insome embodiments, some coherent combinations of received signals may beperformed electronically (i.e., by combining analog signals) beforedigitizing the received signals.

The first combined voxel impression may be combined with the secondcombined voxel impression. If the total aperture defined by the firstreceive aperture and the second receive aperture is greater than a totalcoherent width for the imaging application, then the first combinedvoxel impression may be combined incoherently with the second combinedvoxel impression to obtain a final representation of the selected voxel.

These steps of combining impressions of a selected voxel may be repeatedor performed in parallel for each voxel of the imaged three-dimensionalvolume to obtain a final representation of the entire volume. In otherembodiments, the steps may be performed in any other sequence, with anynumber of transmitted pings, and with any number of receive apertures.Various other combinations of coherent and incoherent summationtechniques may also be used when combining signals from multiple receiveelements, receive apertures, and/or pings.

In some embodiments, after the above example process or another processhas been used to form a complete representation of the 3D volume, asingle plane may be selected for display by identifying a collection ofvoxels making up the selected plane, and displaying the data from thosevoxel on a two-dimensional display.

In other embodiments, a selected two-dimensional plane may be beamformeddirectly from volumetric raw data instead of beamforming a complete 3Dvolume. This may be desirable in order to reduce a quantity ofprocessing needed to produce an image of a selected plane.

FIG. 1 illustrates an example embodiment of a process 10 for beamforminga two-dimensional plane from three-dimensional data obtained by aping-based multiple aperture imaging system. As shown at block 12,imaging signals may be transmitted into a three-dimensional volume. Atblock 14, signals from the transmitted signals may be received byreceive elements of the imaging probe. Block 16 may comprise digitizingsignals received by each receive transducer element of the probe aselement-specific raw data sets (i.e., complete echo stringscorresponding to each receive channel as described elsewhere herein). Atblock 18, a two-dimensional image plane within the insonifiedthree-dimensional volume may be identified manually by a user orautomatically by an imaging device or other system. At block 20,three-dimensional voxels intersecting the selected image plane may beidentified. At block 22, the process may include identifying portions ofeach of the complete element-specific raw data sets corresponding to theidentified voxels. The portions of the data sets may comprise completesamples and/or interpolated positions between samples. Identifying datasamples corresponding to specified voxels may be performed based onknown positions of transmit and receive elements and a speed-of-soundvalue (which may be based on an ultrasound frequency and the compositionof an imaged medium) as described above. At block 24, the process mayproceed by selecting only the identified samples for beamforming todetermine display values for each pixel of the selected two-dimensionalimage plane.

FIG. 2 illustrates a schematic representation of an examplethree-dimensional multiple aperture ultrasound imaging probe array 100(probe housing and support structures are omitted from the drawing forsimplicity) comprising an array of transducer elements and a region ofinterest 120 to be imaged represented as a rectangular block (an actualshape and size of an insonified region may depend on details of theprobe being used). The probe arrays 100 of FIG. 2 and FIG. 3 are shownas having curvature about two orthogonal axes, thereby forming athree-dimensional array with all elements spaced from one another in atleast two dimensions. In alternative embodiments, a probe array may besubstantially flat with all elements lying in substantially the sametwo-dimensional plane. In further embodiments, any other configurationis also possible. For example, some elements may lie on a common planewhile others may be angled inwards or outwards relative to an object tobe imaged. As will be clear in view of the disclosure herein, as long asthe position of each element is known to a desired degree of precisionand accuracy, any array shape may be used, though some arrayconfigurations may be more optimally configured for a particular imagingapplication.

The probe array 100 is shown with a plurality of transmit elements T₁,T₂, and T_(n) highlighted. In some cases transmit elements may bededicated for transmit only, while in other cases, any of the transducerelements may be temporarily designated as a transmit element for aparticular image cycle or ping cycle. In some embodiments, any elementof the array may be temporarily or permanently designated and used as atransmit element. In other embodiments, transmit elements may beconfigured differently than receive elements and/or may be usedexclusively for transmitting. Transmit elements may be located at anyposition within a two-dimensional or three-dimensional array.

In some embodiments, some or all elements of the array 10 may beconfigured to receive echoes of transmitted signals. Such receiveelements may be grouped into a plurality of receive apertures, eachreceive aperture comprising one or more receive elements as describedherein. Grouping of elements into receive apertures may be performed atany time before or after imaging is performed. Furthermore, using storedraw echo data, receive apertures may be re-defined after collecting echodata, as described in further detail below.

FIG. 2 shows two receive apertures R₁, R₂, and R₃. As shown, R1 is madeup of more elements than R2. It should be understood that each of thereceive apertures may include any number of transducer elements whichmay be spaced from one another in one, two or three dimensions. Theelements of the probe array may be grouped into any number of receiveapertures as needed. Because echoes of each ping may be received by allor substantially all of the receive elements, and raw echo data fromechoes received by each element may be digitized and stored in a rawdata memory, the grouping of receive elements into receive apertures maybe established or adjusted prior to imaging, during live imaging, orduring subsequent review of stored raw data in order to optimize thearrangement of apertures for a given imaging scenario.

In some embodiments, the size of a receive aperture may be limited bythe assumption that the speed of sound is the same for every path from ascatterer to each element of the receive aperture. In a narrow enoughreceive aperture this simplifying assumption is acceptable. However, asreceive aperture width increases, an inflection point is reached(referred to herein as the “maximum coherent aperture width,” “maximumcoherent width” or “coherence width”) at which the echo return pathswill necessarily pass though different types of tissue having differentspeeds of sound. When this aggregate difference results in phase shiftsapproaching 180 degrees, additional receive elements beyond the maximumcoherent receive aperture width will tend to degrade the image ratherthan improve it.

Therefore, in order to make use of a wide probe with a total aperturewidth greater than the maximum coherent width, the full probe width maybe physically or logically divided into multiple apertures, each ofwhich may be limited to a maximum width (e.g., a circular diameter, anellipse major axis length, or a rectangular/square aperture's diagonallength) no greater than the maximum coherent aperture width for anintended imaging application (that is, small enough to avoid phasecancellation of received signals). The maximum coherent width can bedifferent for different patients and for different probe positions onthe same patient. In some embodiments, a compromise width may bedetermined for a given imaging scenario. In other embodiments, amultiple aperture ultrasound imaging control system may be configuredwith a dynamic control mechanism to subdivide the available elements inmultiple apertures into groups that are small enough to avoiddestructive phase cancellation. Determining such a maximum aperturewidth may be achieved by sequentially evaluating images, image data, orother data produced using incrementally larger apertures until phasecancellation is detected, and then backing up by one or more aperturesize increments.

In some embodiments, it may be more difficult to meet design constraintswhile grouping elements into apertures with a width less than themaximum coherent width. For example, if the material being examined istoo heterogeneous over very small areas, it may be impractical or toocostly to form apertures small enough to be less than the maximumcoherent width. Similarly, if a system is designed to image a very smalltarget at a substantial depth, an aperture with a width greater than theaccepted maximum coherent width may be needed. In such cases, a receiveaperture with a width greater than the maximum coherent width can beaccommodated by making additional adjustments, or corrections may bemade to account for differences in the speed-of-sound along differentpaths, allowing the region just enveloping the very small, very deeptarget to be brought into precise focus while other regions may beslightly defocused. Some examples of such speed-of-sound adjustments areprovided here, while other methods may also be known.

Because ping signals insonify an entire region to be imaged, volumetricecho data obtained via three-dimensional ping-based imaging is seamless.By contrast, volumetric data assembled from a series of 2D planar slicestend to require some amount of interpolation of image data in the spacesbetween adjacent planar slices. Similarly, individual 2D imagesassembled from a series of scanlines typically require some amount ofinterpolation of image data in the spaces between adjacent scanlines.

The seamless nature of ping-based volumetric echo data means that anyarbitrary 2D slices taken through any portion of a 3D volume may bebeamformed and displayed without the need for interpolation. In somecases, non-planar or curved slices may also be taken through a sectionof volumetric data. The result of such a non-planar or curved-path slicemay be displayed on a two-dimensional display, either as a flattenedplanar image or as a perspective rendering. Volumetric information mayalso be presented via a three-dimensional display such as a holographicdisplay or a stereoscopic display. Therefore, in some embodiments, rawecho data from a volumetric imaging session may be retrieved from amemory device, some or all of the volume may be beamformed and displayedas an image, a desired region of the volume may be selected(automatically by software or manually by an operator), and the selectedregion may be re-beamformed and presented as a new image. Volumetric rawecho data may also be used in a wide range of other ways, as describedbelow.

FIG. 3 illustrates a schematic probe array 100 highlighting a pluralityof transmit elements T1, T2, T3, T4, T5, and two receive aperture groupsof elements R3 and R4. FIG. 3 also shows ray lines indicating pathstraveled by ultrasound energy transmitted by each transmit element T1,T2, T3, T4, T5, to a reflector 140 within a single 2D plane 150 anddashed ray lines representing the paths traveled by echoes reflected bythe reflector 140 and received at each of the receive apertures R3 andR4. As can be seen, although the transmit elements and the receiverelements do not all lie along a common plane, reflectors lying withinthe indicated 2D plane 150 can be illuminated with ultrasound energyfrom any of the transmit elements, and echoes may be received by receiveelements located anywhere in the probe array 100.

Thus, even using a volumetric probe, reflectors lying along a singletwo-dimensional plane (e.g., plane 150) may be selected for beamformingand display from within an insonified three-dimensional volume. The useof transmitters and receivers not on the image plane makes possiblepoint spread functions that are much smaller in the dimensionperpendicular to the image plane than point spread functions in the sameplane obtained with a 2D probe (i.e., a probe configured to transmit andreceive energy focused within the image plane).

As will be described in further detail below, beamforming of receivedecho data may be performed in real-time during a live imaging session,and/or at a later time by retrieving raw echo data of an imagingsession. Depending on a probe used, and the needs of a particularapplication, raw echo data sets may be collected and stored for imagingsessions covering one or more individual two-dimensional planes, or forcomplete three-dimensional volumes.

Raw Echo Data

FIG. 4 is a block diagram illustrating components that may be includedin some embodiments of an ultrasound imaging system 200. The diagram ofFIG. 4 includes several subsystems: a transmit control subsystem 204, aprobe subsystem 202, a receive subsystem 210, an image generationsubsystem 230, and a video subsystem 240. Unlike most ultrasoundsystems, the system of FIG. 4 provides a memory device configured tostore raw, un-beamformed echo data for later retrieval and processing.In some embodiments, the various subsystems may be physically andlogically contained within a single device. In other embodiments, someor all of the subsystems may be contained in physically separate devicesor systems that may be in communication with other devices containingsome or all of the other subsystems. Additional details of the elementsof FIG. 4 are described in Applicant's U.S. patent application Ser. No.13/971,689, published as US Publication No. 2014/0058266 (now U.S. Pat.No. 9,986,969), the entirety of which is incorporated by referenceherein.

The transmit control subsystem may generally comprise controlelectronics for determining the shape, timing, frequency or othercharacteristics of transmitted ultrasound pulses. The probe subsystemmay include any probe configured to transmit ultrasound energy into amedium to be imaged and to receive echoes of the transmitted energy fromwithin the medium. In some cases, transmit and receive functions may bedivided into physically, electronically, and/or logically separatedevices, and in some cases, a receive probe may receivedirectly-transmitted energy in addition to reflected energy. In somecases, one or more elements of the imaging system 200 of FIG. 4 may beomitted.

The receive subsystem 210 may generally include a plurality of separatechannels (e.g., one channel per receive transducer element, in someembodiments), each channel having an Analog Front End (AFE) 212configured to perform various amplifying, filtering and other handlingof analog signals from the probe subsystem's receive transducerelements. The AFE may be connected to an Analog-to-Digital conversiondevice/system (ADC) 214 which may be configured to convert receivedanalog signals into digital signals. Such digital signals may be storedin a digital memory device such as a raw data memory device 220 asdescribed below, and/or optionally transmitted directly 250 to elementsof an image generation subsystem 230.

The image generation subsystem 230 may include a beamformer block 232and, in some cases, may also include an image-layer combining block 234and/or other processing blocks. The image generation subsystem 230 maygenerally be configured to convert the digital raw echo data receivedfrom the receive sub-system 210 or the raw data memory 220 into a seriesof displayable images. In some embodiments, the displayable imagesproduced by the image generation subsystem 230 may be stored in an imagebuffer storage device 236. The video/image display subsystem 240 mayinclude components such as a video processor block 242 configured toconvert the series of displayable images from the image generationsubsystem 230 into an analog or digital video stream that may bedisplayed on an output display device 244. The video/image displaysubsystem 240 may also include an analog or digital memory deviceconfigured to store video streams for display at a different time and/orat a different location. The image generation subsystem may also includea video memory device 246 configured to store beamformed and processeddigital video files.

Any of the digital storage devices described herein, such as raw datamemory devices, video memory devices, image memory devices, datawarehouses and others may include any number of any suitablenon-volatile digital memory device or combinations thereof. Examples ofdigital storage devices may include hard disk drives, solid state diskdrives, flash memory devices, other solid state removable non-volatilestorage devices such as SD cards or USB flash memory devices, opticalstorage devices such as a CDs DVDs, or Blu-Ray, magnetic tape, or anynon-volatile digital memory device. In some cases, analog storagedevices may also be used for data storage.

As used herein, the phrases “echo data,” “raw echo data” and “raw data”may refer to stored echo information describing received ultrasoundechoes at any level of processing prior to beamforming. In variousembodiments, received echo data may be stored at various stages betweenpure analog echo signals to fully processed digital images or evendigital video. For example, a raw analog signal may be stored using ananalog recording medium such as analog magnetic tape. At a slightlyhigher level of processing, digital data may be stored immediately afterpassing the analog signal through an analog-to-digital converter.Further incremental processing, such as band-pass filtering,interpolation, down-sampling, up-sampling, other filtering, etc., may beperformed on the digitized echo data, and “raw” output data may bestored after such additional filtering or processing steps. Such rawdata may then be beamformed to determine a pixel location for eachreceived echo, thereby forming an image. Individual still images may becombined as frames to form motion video. In some embodiments of thesystems and methods described herein, it may be desirable to storedigitized raw echo data after performing very little processing, e.g.,after some filtering and conditioning of digital echo data, but beforeperforming any beamforming or image processing.

Although the term “echo data” is generally used herein to refer to datareceived by receive elements, the term “echo data” is also intended toinclude data generated by digitizing received signals resulting fromdirect transmission of ultrasound or other transmitted energy signalswithout necessarily being reflected. Therefore, the phrase “echo data”may generally have the same meaning as “receive data.”

In addition to received echo data, it may also be desirable to storeinformation about one or more transmitted ultrasound signals thatgenerated a particular set of echo data. For example, when imaging witha multiple aperture ping-based ultrasound method as described above, itis desirable to know information about a transmitted ping that produceda particular set of echoes. Such information may include the identityand/or position of one or more transmit elements, as well as frequency,amplitude (magnitude), pulse length (duration), waveform (shape), orother information describing a transmitted ultrasound signal.

Transmit data may be collectively referred herein to as “TX data”. Insome embodiments, such TX data may be stored explicitly in the same rawdata memory device in which raw echo data is stored. For example, TXdata describing a transmitted signal may be stored as a header before oras a footer after a set of raw echo data generated by the transmittedsignal. In other embodiments, TX data may be stored explicitly in aseparate memory device that is also accessible to any system performinga beamforming process (e.g., a PC, laptop, tablet, mobile device,server, imaging system, or other suitably configured device). Inembodiments in which transmit data is stored explicitly, the phrases“raw echo data” or “raw data” may also include such explicitly stored TXdata.

TX data may also be stored implicitly. For example, if an imaging systemis configured to transmit consistently defined ultrasound signals (e.g.,consistent amplitude, waveform shape, frequency, pulse length, etc.) ina consistent or known sequence, then such information may be assumedduring a beamforming process. In such cases, the only information thatneeds to be associated with the echo data is the position (or identity)of the transmit transducer(s). In some embodiments, such information maybe implicitly stored and extracted based on the organization of raw echodata in a raw data memory.

For example, a system may be configured to store a fixed number of echorecords following each ping. In such embodiments, echoes from a firstping may be stored at memory positions 0 through ‘n−1’ (where ‘n’ is thenumber of records stored for each ping), and echoes from a second pingmay be stored at memory positions n through 2n−1. In other embodiments,one or more empty or specially encoded records may be left in betweenecho sets. In some embodiments received echo data may be stored usingany of various memory interleaving techniques to imply a relationshipbetween a transmitted ping and a received echo data point (or a group ofechoes). In general, a collection of data records corresponding toechoes or other signals resulting from a single transmitted pingreceived by a single receive element may be referred to herein as asingle “echo string.”

A “complete echo string” may refer to substantially all data resultingfrom a single ping received by a receive element, whereas a “partialstring” or a “partial echo string” may refer to a sub-set of all echoesof the single ping received by the receive element.

Similarly, a “complete data set” may refer to substantially all raw data(e.g., echoes or directly-received signals) resulting from a defined setof transmitted signals. A set of transmitted signals may be defined asan identifiable set of transmitted pings, as all pings or other signalstransmitted within a defined period of time, or otherwise. A “partialdata set” may refer to a sub-set of all raw data resulting from thedefined set of transmitted signals.

In some cases, a complete echo string or a complete data set maycomprise less than all theoretically available data, because some datamay be discarded as undesirable. For example, data representing a firstfew milliseconds following transmission of a ping may containsubstantial cross-talk or other noise that may not meaningfullycontribute to a desired dataset, and may therefore be ignored.Nonetheless, the resulting dataset may still be considered a “completeecho string” or a “complete data set” if it contains all of the desireddata resulting from a transmitted ping (or a set of pings). Partial echostrings or partial data sets may be obtained by selecting a sub-set ofrecords from a complete echo string or data set in order to limit a dataset for the purposes of faster data communication or limiting processingresources, for example.

Similarly, assuming data is sampled at a consistent, known samplingrate, the time at which each echo data point was received may beinferred from the position of that data point in memory. In someembodiments, the same techniques may also be used to implicitly storeand organize/interpret data from multiple receive channels in a singleraw data memory device.

In other embodiments, the raw echo data stored in the raw data memorydevice 220 may be physically or logically located in any other structureas desired, provided that the system retrieving the echo data is able todetermine which echo signals correspond to which receive transducerelement and to which transmitted ping. In some embodiments, positiondata describing the exact physical location of each receive transducerelement relative to a common coordinate system may be stored in thecalibration memory device 238 along with information that may be linkedto the echo data received by that same element. Similarly, position datadescribing the exact physical location of each transmit transducerelement may be stored in the calibration memory device 238 along withinformation that may be linked to TX data describing each pingtransmitted from that transmit element.

In general, calibration data describing position and/or performanceinformation about each transducer element may be physically located inany device electronically accessible by the device performingbeamforming operations. For example, calibration data may be located ina probe device itself, in an imaging system connected by a wired orwireless connection to the probe, in a network-accessible databaseaccessible by an imaging system, or by a server or other computingdevice configured to perform beamforming operations.

In some embodiments, calibration data may also include performanceinformation. Performance information may include information identifyingelements that have become damaged to the point that they provideresponse data that does not accurately describe the echoes impinging onthe element. Depending on the nature of the inaccurate information, datafrom damaged elements may be ignored or weighted to minimize detrimentaleffects on resulting images.

In some embodiments, additional information useful in beamforming imagesbased on stored echo data may also be stored in a digital storage deviceaccessible to a device performing beamforming operations. Examples ofsuch additional information may include speed-of-sound values, such asaverage speed-of-sound values, path-specific speed-of-sound values(e.g., a speed of sound along a ray path from a transmit aperture to apixel/voxel location to a receive aperture), receive-aperture-specificspeed-of-sound values, or others. Additional stored information may alsoinclude weighting factors or user-controllable settings used during adata-capture session.

In some embodiments, each echo string in the raw data memory device 220may be associated with position data describing the position of thereceive transducer element that received the echoes and with datadescribing the position of one or more transmit elements of a transmitaperture that transmitted the ping that produced the echoes. Each echostring may also be associated with TX data describing characteristics ofthe transmitted ping such as power level, frequency, pulse length/signalshape, emitter efficiency, etc. Such associations may be made using anysuitable data structures.

In some cases, raw echo data may also be associated with various“meta-data,” including information allowing a clinician or serviceprovider to associate the raw echo data with a patient, imagingdate/time, imaging location, imaging environment (ambient temperature,humidity, barometric pressure, etc.), imaging system settings usedduring image capture, object surface temperature, or other informationthat may be useful in using the raw data. Any other meta-data may alsobe associated with raw echo data records.

One benefit of storing raw echo data is that the information may beretrieved, processed, and reviewed at a later time, allowing a fargreater degree of control and flexibility than if only a video stream(e.g., a cine loop) were saved from an imaging session. For example, inone embodiment, a patient may visit a technician and the technician mayconduct an ultrasound examination during which raw echo data is capturedand stored. Hours, days, weeks, or even months later (in other words,any time after the patient's original session), a trained professionalsuch as a physician may use a personal computer, laptop, tablet or animaging system to re-examine a wide range of images derivable from datagenerated during the examination session and to create new images (thatis, images that were not produced during the imaging session with thepatient) by manipulating the raw data without re-examining or re-imagingthe patient. In some embodiments, such re-examination of stored data mayinclude several processes that are only possible with access to raw echodata.

In some embodiments, raw data from an imaging session may be storedalong with raw echo data captured while imaging a calibration phantom.For example, raw echo data obtained while imaging a calibration phantommay be used for later calibration of the imaging session data bycorrecting transducer element position assumptions made during livebeamforming.

Information describing the position of each transducer element may beobtained by a calibration process as described in Applicants' priorapplications. Such element position data may be stored in a calibrationmemory device 220, which may be physically located with otherelectronics, or may be located in a remote, network-accessible server.However, in some embodiments, the element-position information maychange between performing a calibration operation and capturing rawultrasound data. For example, a probe may have been dropped, damaged orotherwise altered before or during a raw echo data capture session.

In some embodiments, the ability to re-process stored raw echo datameans that a probe may actually be retroactively re-calibrated after rawecho data is captured, and the data may be re-beamformed using theupdated element position information.

In other embodiments, raw echo data stored in a raw data memory devicemay be analyzed to determine whether a probe is actually out ofcalibration.

Raw Data Capture Devices

In various embodiments, a network-based imaging system may provide theadvantage of de-coupling the collection of raw echo data from formationand display of images derived from the collected raw echo data. As aresult, systems operating as components in a network-based imagingsystem may be configured to operate in two broadly-defined modes. In a“live imaging” or “real-time imaging” mode, the system may be configuredto process and display images based on echo data with as little latencyas possible. Latency may be defined as the time delay between when anaction (such as moving the probe relative to the object being imaged)occurs and when the imaging system displays a result of the action.Various examples of live-imaging modes are described below.

A second broad mode may be described as a “high quality data capture”mode. When a “high quality data capture” mode is initiated, a datacapture component of a network-based imaging system may collect andstore raw echo data (along with TX data, and other data as describedherein) from a predetermined time period, number of ping cycles, ornumber of image cycles. During a high quality data capture mode, thefull data set need not be beamformed or otherwise processed in realtime. In some embodiments, the data capture device may store the rawdata in an external (e.g., network-connected) storage device inreal-time as it is captured. In other embodiments, such as whenreal-time network communication resources are limited, the data capturedevice may store the raw data in a local storage device in real-time,and may subsequently transfer the captured data to an external (e.g.,network-connected) storage device at a later time when network resourcesare less constrained. A single remote data storage device or acollection of remote data storage devices may be referred to herein as a“data warehouse,” which may include any number of network-connected datastorage devices as needed.

Embodiments of network-based imaging systems may generally include datacapture devices having lower cost and limited-performance hardware forperforming transmitting, receiving and data storage functions. Datacapture devices may be physically de-coupled from and located physicallyremotely from image generation devices which may have higher cost andhigher performance hardware and/or software for performing beamformingand image processing functions. Some embodiments of network-basedimaging systems may also include end-use viewer terminals innetwork-communication with the image generation devices and/or incommunication directly with one or more data capture devices. Suchviewer terminals may be used for real-time or time-shifted viewing ofimages generated from captured raw data. In some embodiments, datacapture devices may be configured for operation by relativelyminimally-trained technicians who may be guided by one or more highlytrained professionals via network communications or by a software orartificial intelligence agent.

In some embodiments, raw echo data that is captured and stored in a rawdata memory device as described above may subsequently be copied,forwarded, or otherwise electronically communicated to an external(e.g., a backup) memory storage device. Such data communications maytake place over any available wired or wireless data transfer system,such as Bluetooth, IR/Infra-Red, USB, IEEE 1394 Firewire, Thunderbolt,Ethernet/Intranet/Internet (TCP/IP, FTP, etc.) or others.

In some embodiments, the raw data may be loaded back onto an ultrasoundimaging system (e.g., the same system originally used for insonificationand raw echo data capture), or a similarly-configured ultrasound imagingsystem for re-processing, re-beamforming, and image generation/viewing.In other embodiments, a personal computer, laptop, tablet, mobiledevice, network-connected server, or other digital computing device maybe configured with software and/or hardware to beamform and/or processthe raw echo data into images without the use of a dedicated ultrasoundimaging system.

In other embodiments, raw echo data may be beamformed, processed anddisplayed by software on any other suitably configured computationaldevice or system, such as a tablet or smart phone. In other embodiments,raw echo data may be uploaded over a network to a network-accessibleserver which may store and process image data remotely.

FIG. 5 illustrates an embodiment of an imaging system 201 divided intoan energy (e.g., ultrasound) data capture and communication device 260and a remote image generation and display system 262. The data captureand communication device 260 may be configured with minimal hardwarecomponents for communication of raw echo data to the remote imagingsystem 262 via a communications device 264 and a wired or wirelessnetwork 266.

The data capture device 260 of FIG. 5 may include a probe 202, atransmit controller 204, an AFE 212 and an ADC 214 as described above.In place of any beamforming or image processing components, the datacapture device 260 may instead include a communications device 264configured to communicate raw echo data to a remote system 262 via anetwork 266. The remote system 262 may include hardware, firmware and/orsoftware configured to beamform and process the raw echo data capturedby the device 260.

In some embodiments, the probe 202 may be an ultrasound probe withultrasound transducer elements spaced from one another in two or threedimensions and configured for capturing 3D volumetric ultrasound data asdescribed herein.

In some embodiments, the communications device 264 may be configured tostream raw echo data in real time to the remote system 262. In otherembodiments, the data capture device 260 may include an internal memorydevice 220 for short term storage of raw echo data (e.g., as acommunication buffer). In other embodiments, an internal memory device220 within the data capture and communication device 260 may beconfigured for longer term storage of raw echo data within the capturedevice 260. In further embodiments, a data capture device may containone, two or more memory devices 220 configured for various uses.

For example, a data capture and communication device 260 may include afirst memory device 220 configured to operate as a circular buffer forstoring real-time data immediately prior to communicating the real-timedata over a network 266. After data has been communicated from thedevice 260, the data may be deleted or overwritten with newly acquireddata. The data capture device 260 may also contain a second memorydevice configured to operate as a circular buffer for a full set of rawecho data to be communicated over the network in response to a commandfrom an operator.

In some embodiments, a system 201 such as that shown in FIG. 5 may beused in an environment in which an operator of the data capture device260 does not require a display, such as when using a probe configured tobe placed in a stationary position on a patient during a data capturesession. In some cases, a third party (e.g., in addition to the patientand the data capture device operator) may view real-time images producedfrom the raw echo data obtained by the data capture device 260 andcommunicated over the data network 266. The third party reviewing theimages may then provide real-time instructions relating to the placementof the probe on a patient or other object to be imaged. In someembodiments, such positioning instructions may be delivered verbally,such as over a telephone or other audio connection. Alternatively, probeplacement instructions may be communicated to the operator by way ofindicators on the probe itself or by an external device such as adisplay screen or other device.

For example, a tablet device with an integrated camera may be used toproduce an optical image of the patient or other object with the probe202 in place. The tablet may include an application configured toindicate a direction and distance of movement of the probe on thepatient (or other object) to a more ideal location. Such movementinstructions may be provided by a third-party viewing images at a remoteimage generation and display system 262. Alternatively, probepositioning instructions may be provided to the operator by anartificial intelligence application on the tablet or by directionalindicators on an image being displayed on a tablet or other displayscreen.

In some embodiments, some or all elements of a remote image generationand display system 262 may be implemented in a tablet, personalcomputer, laptop, mobile device, or a combination of such elements,collectively referred to herein as a “control surface”. For example, insome embodiments, image generation, raw data storage and image bufferingfunctions may be performed in a computing device which may be in wiredor wireless communication with a control surface such as a handheldtablet device which may be configured to perform the display and videoprocessing functions along with user interface functions.

FIG. 6 illustrates another example of an imaging system 300 comprising alimited function data capture and communication device 310, anetwork-based beamforming and video processing device 320, and one ormore viewer terminals 330 connected to one another via a wired orwireless data network. As shown in the illustrated example, the datacapture and communication device 310 may include a probe 202, a transmitcontroller 204, an AFE, 212, an ADC 214, a raw-data memory device 220, acommunications device 264, and a display 244. In some embodiments, theprobe 202 may be an ultrasound probe with ultrasound transducer elementsspaced from one another in two or three dimensions and configured forcapturing 3D volumetric ultrasound data as described herein.

The network-based beamforming and video processing device 320 mayinclude one or more digital storage devices comprising a raw datawarehouse 221, an image generation subsystem 230 which may includehardware and/or software for performing beamforming, image layercombining (as described above, for example) and other image generationprocesses. The beamforming and video processing device 320 may alsoinclude a calibration memory 238, an image buffer 236, a video processor242, and a video memory 246.

In operation, the imaging system 300 may be used for live real-timeimaging of a patient or object to be imaged. An example of a real-timeimaging process is described below with reference to FIG. 7. Livereal-time imaging may be performed using the limited function datacapture and communication device 310 by using the probe 202 to transmitultrasound pulses into the region of interest (such as unfocusedthree-dimensional pings) and receive echo signals from the region ofinterest. The received signals and other information (e.g., calibrationinformation, TX data, device identifier, etc.) may be communicated fromthe data capture and communication device 310 to the beamforming andvideo processing device 320 over the network 266. The beamforming andvideo processing device 320 may beamform the echo data to generateimages and may produce data representing a video stream that may beelectronically communicated over the network 266 back to the datacapture and communication device 310 for display to an operator on thedisplay 244 of the device 310.

The data capture and communication device 310 of FIG. 6 may include aprobe configured to transmit into and receive energy from an entirethree-dimensional volume to be imaged. Alternatively, the data captureand communication device 310 of FIG. 6 may include a probe configured toinsonify and receive echoes from only a single imaging plane. In someembodiments, the data capture and communication device 310 of FIG. 6 maybe configured with beamforming hardware and software omitted. This mayallow for the data capture and communication device 310 to beconstructed at relatively lower cost, and utilize components withrelatively lower power demand. In some embodiments, several suchlow-cost data capture and communication devices 310 may be deployedwithin a local network (such as a hospital, medical center, imagingcenter, or other facility in which imaging may be performed). All suchdevices may utilize the same beamforming and video processing device 320over the local network. In some embodiments, the beamforming and videoprocessing device 320 may comprise several servers in order to manage aload of several simultaneous live imaging sessions.

In order to conduct live, real-time imaging sessions using anetwork-based imaging system, it may be desirable to limit the quantityof raw echo data to be processed to form real-time images. The quantityof raw echo data to be processed and/or communicated over a network maybe reduced by using one or more of various data reduction methods. Someexamples of which are provided below.

In the case of a network-based beamformer such as that illustrated inFIG. 5 and FIG. 6, various data reduction methods may be used to reducethe quantity of data communicated over the data network 266 from eachdata capture and communication device 310 to the beamforming and videoprocessing device 320.

One example data reduction approach may involve identifying a reducedset of data samples to be processed into images by defining a limitedimage window. In some embodiments, the quantity of raw echo data to beprocessed and/or communicated over a network can be greatly reduced bydetermining a minimum sample window necessary to generate images of adefined image window.

An image window may be defined as a particular two-dimensional plane(possibly as a portion of a three-dimensional volume), qualified by zoomlevel, and further constrained by left-right pan and up-down elevationwithin the insonified object's region of interest. An image window maybe selected automatically by the data capture and communication device310, manually by an operator of the device, or a combination of manuallyand automatically.

As used herein, the term “sample window” may refer to a range or list ofsample index values identifying a set of stored data samples meetingsome criteria. For example, a sample window may be defined as the set ofdata samples corresponding to pixels of a two-dimensional image windowwhich may be defined in terms of size, position, and orientation withinan insonified volume.

In some embodiments, a process for reducing a data set by image windowsample selection may comprise the steps of (1) defining an image window,(2) identifying receive data samples corresponding to the defined imagewindow, and (3) selecting only those samples corresponding to thedefined image window for processing or communication over a network.

In some embodiments, an optimum set of raw data samples (earliestthrough latest per ping for each receive element) needed for generatinga particular image window may be determined by calculating the samplenumbers (or other sample-identifying indices) corresponding to the topand bottom rows of image window pixels, and communicating only thosereduced ranges of samples to the remote beamforming and video processingdevice 320; all raw data samples outside of these ranges will not beused for beamforming that particular image window, so need not becommunicated to the remote beamforming and video processing device 320.Typically, and depending on zoom level selected, only a quarter or fewerof the total samples collected per ping for each receive element may beused during the beamforming process. Each TX-RX pair may generate aslightly different range of necessary samples for a given image window,but the variation among all pairs may be small enough to simply use theminimum ‘early’ sample number/index through the maximum ‘late’ samplenumber/index across all pings and receive elements.

One data reduction technique may include using a process for directlybeamforming a two-dimensional image from raw data produced byinsonifying a three-dimensional volume, as described above withreference to FIG. 1. In such embodiments, a reduced data set maycomprise only the data corresponding to the selected two-dimensionalimage plane, while a “complete” raw data set may comprise echo datareceived from the entire three-dimensional volume.

A set of data samples to be processed or communicated over a network maybe further reduced by effectively reducing a frame rate of a displayedvideo stream. Frame rate reduction may be performed in multiple ways.For example, in some embodiments a frame rate may be reduced byselecting only echoes of selected pings for processing into imageframes. In other words, a size of a raw data set may be reduced by usingthe data produced by less-than-all transmitted pings. For example, ifonly the echoes of every-other transmitted ping are selected forprocessing, then the data set may be reduced by half. In other examples,selected data may be limited to data received from every third ping,every fourth ping, every fifth ping, etc.

In another example, raw data may be selected based on the positionand/or identity of transmit elements that produced the data. Forexample, if a probe contains X transmit apertures (or transmit elementswhere each transmit aperture has only one element), each of whichtransmits a ping during a typical imaging cycle, then a data set to beprocessed or communicated over a network may be reduced by selectingonly echo data samples corresponding to pings transmitted by X/2, X/3,X/4, X/5, etc. of the transmit elements. In some embodiments, thetransmit elements from which echo data is to be selected may be chosenbased on position of the elements in the probe. For example, if only asmall region under the probe is of interest, then chosen transmitelements may be limited to those above the small region of interest.

In other embodiments, a data set may be reduced by pre-combining echodata before processing or communicating the data over a network. Forexample, receive data received by the same receive element in responseto two separate pings may be combined with one another coherently,thereby reducing two data points to one. In some embodiments, a firstecho string received by a first receive element resulting from a firstping may be coherently combined with a second echo string received bythe first receive element resulting from a second ping. In someembodiments echoes of two or more pings received by the same element maybe combined before performing other data reduction methods.

Many other methods may be used for reducing the set of raw echo data tobe transferred when conducting live, real-time imaging sessions using anetwork-based beamformer and image generation device 320. In oneexample, the real precision of the A/D Converter 214 may be closelymeasured, and the LSB (Least Significant Bit) bits that correspond mostclosely to sampling noise, conversion noise, or quantization error maybe stripped or removed from the data to be transferred. For example, fora 16-bit ADC with sufficiently high statistical error probabilities inbits 0 through 2, it may be sufficient to only communicated bits 3through 15 and pack sequential samples accordingly, reducing bandwidthneeds by nearly 20%.

Another data reduction method may include reducing the frame rate ofcommunicated data to a small fraction of the full frame rate supportedby the Analog Front End (AFE) or other electronics, with a correspondinglinear reduction in bandwidth needs.

Another data reduction method may include, prior to communicating areduced set of data samples over the network 266, compressing the rawdata samples of the set using a lossless or lossy compression algorithmas requirements dictate, allowing for another potential 25% to 75% orgreater reduction in bandwidth needs.

Another data reduction method may include reducing a total number ofreceive elements for which data is selected. For example, a subset ofall receive elements may be selected, and receive data from only thoseselected elements may be selected for beamforming or communication overa network. In some cases, a subset of receive elements may be selectedbased on position of the receive elements relative to a particularfeature of interest within an insonified volume. In other cases, thenumber of elements assigned to an aperture may be reduced, such as byignoring some receive elements in between selected elements, using onlydata from the selected elements.

In other embodiments, entire apertures may be ignored, such as byselecting some apertures to be included in a reduced data set, whileremoving data from one or more apertures. In some embodiments, datareceived by a group of elements between selected apertures may beremoved from a data set to form a reduced data set.

In some cases, several data reduction techniques may be appliedconcurrently, yielding compounded reductions in total communicated data.

In some cases, a level of data reduction to be applied may be based on adesired image quality level for a particular imaging application. Forexample, if a user or an automated system determines that a particularimaging application requires a high quality real-time image, then datareduction methods may be selected based on their impact on image qualityin order to preserve image quality at a minimum level needed for theidentified application. On the other hand, if a lower image quality isacceptable for an identified imaging application, then data reductionmethods that may tend to reduce image quality may be used.

In some embodiments, the data capture and communication device 310 mayinclude a user interface with controls allowing an operator to select animage window, adjust imaging parameters, and capture raw echo data tolocal and/or remote storage for later review and analysis. Capturing rawdata may comprise storing raw echo data received by the transducerelements during several seconds (or more) of imaging. The raw datacaptured may be stored in the raw data memory device 220 in the datacapture and communication block 310, and may be communicated to theremote beamforming and video processing block 320 over the network 266and stored in the raw data warehouse 221.

In some cases, in addition to data reduction, variousprocessing-reduction adjustments may be made to beamforming, image layercombining, or image processing methods. For example, performing fewerdata combining (image layer combining) steps may reduce a quantity ofprocessing needed to produce each image frame. Similarly, by adjusting abalance of coherent vs incoherent summation of image layers, a number ofprocessing cycles to produce an image frame may be increased ordecreased. In other embodiments, any other processing-reduction methodsmay be used.

The raw data captured and stored by the data capture and communicationdevice 310 may include substantially all of the received data, withoutreducing or reducing the data using the techniques described above. Thisallows the raw data set to be used for more detailed review and analysisthan might be available for real-time imaging with a limited-functiondata capture and communication device 310.

In some embodiments, the system of FIG. 6 may be used in combinationwith a viewer terminal 330, such as a laptop computer, a desktopcomputer, a tablet, a smartphone, or other computing device configuredto connect to the remote image generation system. For example, anoperator controlling the probe 202 may view a video stream produced bythe video processing block 320 on a terminal device 330 instead of (orin addition to) the display 244.

FIG. 7 illustrates an example of a network-based imaging process 400that may be performed by a raw data capture device, such as thosedescribed herein (e.g., the device 260 of FIG. 5, the device 310 of FIG.6, the device 312 of FIG. 8, or any other suitably configured device).In various embodiments, the steps of the process 400 of FIG. 7 may beperformed by two or more devices.

The process 400 of FIG. 7 may generally be configured to perform a liveimaging process while communicating a limited data set to a remoteserver (e.g., 262 of FIG. 5, 320 of FIG. 6, or 322 of FIG. 8) which mayperform beamforming operations and communicate a video stream back tothe data capture device and/or an adjacent display device (such as alaptop, PC, tablet, mobile device, or other control surface). In variousembodiments, user interface elements may be provided in the data capturedevice, in a separate control surface device, or in another device.

The process 400 of FIG. 7 may include transmitting and receiving signalswith an imaging probe as shown in block 410. In some embodiments, theoperations of block 410 may include transmitting and receiving steps asdescribed herein with reference to ping-based multiple aperture imaging.Alternatively, transmitting and receiving signals may include any othersuitable imaging process. Any suitable probe device may be used.

At block 412, the process 400 may include digitizing received signals asraw data sets and storing complete raw data sets in a local memorydevice. As described herein, digitizing received signals may beperformed with any suitable analog front end, analog-to-digitalconversion, and/or other hardware and software components. The localmemory device may be any suitable volatile or non-volatile memorydevice.

As described above, a “complete data set” may refer to substantially allraw data (e.g., echoes or directly-received signals) resulting from adefined set of transmitted signals. A set of transmitted signals may bedefined as an identifiable set of transmitted pings, as all pings orother signals transmitted within a defined period of time, or otherwise.Thus, for example, a complete data set may include all digitizedreceived signals resulting from a set of X transmitted pings, where X isany number from 1 to 1 million or more (practical systems may definesets of pings based on a number of transmit elements of a probe, forexample).

At block 414, the process 400 may include selecting portions of thecomplete stored raw data set for communication over the network.Selecting portions of the data sets may include any one or more of thedata reduction methods described above. For example, in someembodiments, a selected reduced data set may comprise only datacorresponding to a selected two-dimensional image plane, while a“complete” raw data set may comprise echo data received from the entireinsonified three-dimensional volume.

At block 416, the process 400 may include communicating the selected rawdata set potions over a network to a remote server. As described herein,the network may include any data network, and the remote server mayinclude any suitable server device. In some embodiments, the remoteserver may be physically located a distance of several miles or morefrom the probe, while in other embodiments, the remote server may belocated in the same room. The server may only be “remote” in the sensethat it is not housed in the same device as the data capture device.

The remote server may process the received information and beamform theraw data set portions to produce images which may be combined to form avideo stream. The methods used by the remote server may include anybeamforming, image layer combining, and/or other image processingtechniques appropriate for the received data, including the variousexample methods described herein.

At block 418, the process 400 may include receiving a video stream fromthe remote server over the network. The video stream may be communicatedusing any suitable digital video communication protocols or methods. Atblock 420, the received video stream may be displayed to an operator ofthe imaging probe (who may also be operating the data capture device).

At block 422, the process 400 may include, in response to a user commandto initiate a “high quality data capture mode” as described herein,communicating a complete raw data set over a network to a remote datastorage device. As described in various examples above, the remote datastorage device may be integral with or separate from the same remoteserver used to perform beamforming and image processing. The networkused for communicating the complete raw data set may be the same networkor a different network than the one used to communicate the selected rawdata set portions. In various embodiments, the complete data set may beretrieved from the remote data storage device, beamformed and processedinto images in near-real-time (e.g., within second or milliseconds of itbeing received at the remote data storage device) or at any longer timedelay.

In various embodiments, each viewer terminal 330 may include independentuser interface controls configured to independently control imagingparameters. Independently-controlled imaging parameters may include anyitems of user-controllable information affecting an image displayed tothe user. For example, imaging parameters may include beamformingparameters such as speed-of-sound and image window selection, or videoprocessing parameters such as brightness, contrast, video filters, etc.

FIG. 8 illustrates an example of an alternate configuration of animaging system 302 with a limited-function data capture andcommunication device 312 in communication with a network-based remoteimage generation system 322. The data capture and communication device312 may optionally include hardware and software elements to enable thedevice to perform some or all beamforming and image generationoperations locally and to display a limited-quality (and/or lower framerate) real-time image to an operator. The data capture and communicationdevice 312 may also be configured to communicate full-quality raw echodata from the full insonified 3D volume or 2D plane to the network-basedimage generation system 322 for real-time and/or time-shiftedmanipulation and viewing by one or more remote professionals (e.g.,physicians, sonographers, or other trained professionals) using a viewerterminal 330.

The data capture and communication device 312 may include a probe 202(e.g., a 3D imaging probe or a 2D imaging probe as described above), atransmit controller 204, an AFE 212, an ADC 214, a raw data memorydevice 220, a communication device 264, an image generation block 231which may perform beamforming and image layer combining operations, avideo processor 242, and a display 244. In some embodiments, the datacapture and communication device 312 may include components withrelatively low processing power and low power requirements in order toallow an operator to see a limited-quality real-time image, whileallowing for full-quality data to be captured, stored and communicatedto a raw data warehouse 221.

The data capture and communication device 312 of FIG. 8 may include aprobe configured to transmit and receive energy in an entirethree-dimensional volume to be imaged. In some embodiments, the imagegeneration block 231 may be configured to beamform echoes received froma single plane within a three-dimensional insonified volume, whileoptionally (e.g., on an operator's command) storing several secondsworth of raw echo data from the full 3D volume in the on-board raw datamemory 220 and/or communicating the 3D volumetric raw data to the rawdata warehouse 221.

In order to limit the hardware requirements for the data capture andcommunication device 312, on-board real-time beamforming and imagegeneration and display hardware may be limited to generating andprocessing images for one or more two-dimensional slices within aninsonified 3D volume. In some embodiments, the 2D slice (or slices) tobe beamformed and displayed may be selected manually by an operator. Inother embodiments, the displayed image planes may be fixed for aparticular probe, or may be automatically selected by software. Fixedplanes, for example, may include a pair of orthogonal planesintersecting at the center of a probe array (e.g., the plane 150 in FIG.3 and a vertical plane orthogonal to plane 150), plus a horizontal planeorthogonal to the two vertical planes. Fixed planes may include axial,corneal, sagittal, transverse and other planes commonly used inanatomical imaging. These two or more planes may be displayedside-by-side on the display 244. Alternatively, two, three, four or moreuser-selectable (and not necessarily orthogonal) planes may also bedisplayed simultaneously.

Additionally, the amount of processing to be performed by the imagegeneration block 231 may be limited by utilizing any of the datareduction methods described above, such as producing images at a framerate substantially lower than a maximum frame rate that may beachievable with ping-based multiple aperture imaging.

The system of FIG. 8 may also be used in combination with a viewerterminal 330, such as a laptop computer, a desktop computer, a tablet, asmartphone, or other computing device configured to connect to theremote image generation system in order to allow the full-quality imagedata to be viewed by the operator either in real-time or after theimaging session has been completed.

In some embodiments all or a substantial portion of the raw datacollected by a data capture device may be communicated to the remoteimage generation system 322 in real-time (or as close to real-time aspossible). In such embodiments, one or more remote users may view nearreal-time images of the imaging session via a network-connected viewerterminal 330.

The remote image generation system 322 may include a calibration memorydevice 238, a raw data warehouse 221, and an image processing server324. The image processing server may include hardware and softwareelements suitable for performing any of the sub-processes describedherein, such as beamforming, image layer combining, image process, videoprocessing, etc. The image processing server 324 may also be configuredto retrieve raw data from the raw data warehouse and correspondingcalibration data from the calibration memory based on a request from auser operating a viewer terminal or a data capture and communicationdevice 312 for an identified imaging session.

In some embodiments, the remote user may select one or more imagewindows entirely independent of an image window (if any) being viewed byan operator of the data capture and communication device 312. Similarly,the remote user may adjust beamforming and image generation parametersindependently of settings used by an operator of the data capture andcommunication devices 312 without necessarily changing an imagedisplayed to the operator. Variables that may be adjusted by the remoteviewer user may include speed-of-sound values, zoom level, pan window,weighting factors, image layer combining algorithms, etc.

In addition to ping-based beamforming, beamformer technology may beseparately packaged for embedding into a scanline-based imaging systemor installed on a server and made available to scanline-based imagingsystems through a network connection. In some embodiments, a datacapture device may comprise a tap configured to intercept electricalsignals sent to and received from a conventional ultrasound (or other)imaging probe.

FIG. 9 illustrates an example of a network-based imaging process 450that may be performed by a raw data capture device that includes atleast some beamforming and imaging processing electronics, such as thedevice 312 described herein with reference to FIG. 8. In variousembodiments, the steps of the process 450 of FIG. 9 may be performed bytwo or more devices.

The process 450 of FIG. 9 may generally be configured to perform a liveimaging process using a built-in processor to beamform and process alimited data set and display a limited-quality video stream whilecapturing and storing a complete data set from which a full-qualityvideo stream may be produced. On command, the complete data set may becommunicated over a network to a remote storage device. In variousembodiments, user interface elements may be provided in the data capturedevice, in a separate control surface device, or in another device.

The process 450 of FIG. 9 may include transmitting and receiving signalswith an imaging probe as shown in block 452. In some embodiments, theoperations of block 452 may include transmitting and receiving steps asdescribed herein with reference to ping-based multiple aperture imaging.Alternatively, transmitting and receiving signals may include any othersuitable imaging process. Any suitable probe device may be used.

At block 454, the process 450 may include digitizing received signals asraw data sets and storing complete raw data sets in a local memorydevice. As described herein, digitizing received signals may beperformed with any suitable analog front end, analog-to-digitalconversion, and/or other hardware and software components. The localmemory device may be any suitable volatile or non-volatile memorydevice.

As described above, a “complete data set” may refer to substantially allraw data (e.g., echoes or directly-received signals) resulting from adefined set of transmitted signals. A set of transmitted signals may bedefined as an identifiable set of transmitted pings, as all pings orother signals transmitted within a defined period of time, or otherwise.Thus, for example, a complete data set may include all digitizedreceived signals resulting from a set of X transmitted pings, where X isany number from 1 to 1 million or more (practical systems may definesets of pings based on a number of transmit elements of a probe, forexample).

At block 456, the process 450 may include selecting portions of thecomplete stored raw data set for real-time beamforming and imageprocessing. Selecting portions of the data sets may include any one ormore of the data reduction methods described above. For example, in someembodiments, a selected reduced data set may comprise only datacorresponding to a selected two-dimensional image plane, while a“complete” raw data set may comprise echo data received from the entireinsonified three-dimensional volume.

At block 458, the selected raw data set portions may be processed andbeamformed to produce a limited-quality video stream. The imageprocessing block 231 and the video processing block 242 within the datacapture and communications device 312 may beamform and process theselected raw data portions to produce images which may be combined toform a limited quality video stream. The methods used by the imageprocessing block 231 may include any beamforming, image layer combining,and/or other image processing techniques appropriate for the selecteddata, including the various example methods described herein.

At block 460, the limited quality video stream may be displayed to anoperator of the imaging probe (who may also be operating the datacapture and communications device 312).

At block 462, the process 450 may include, in response to a user commandto initiate a “high quality data capture mode” as described herein,communicating a complete raw data set over a network to a remote datastorage device. As described in various examples above, the remote datastorage device may be integral with or separate from the same remoteserver used to perform beamforming and image processing. The networkused for communicating the complete raw data set may be the same networkor a different network than the one used to communicate the selected rawdata set portions. In various embodiments, the complete data set may beretrieved from the remote data storage device, beamformed and processedinto images in near-real-time (e.g., within second or milliseconds of itbeing received at the remote data storage device) or at any longer timedelay.

The retrieved complete data set may be processed by the image processingserver 324 to produce a video stream of higher quality than the limitedquality video stream. The retrieved complete data set may also beprocessed by the image processing server 324 to produce entirelydifferent images and/or video than what was viewed in real-time by theoperator of the data capture device.

Remote-Guided Imaging

In some embodiments, an operator of a limited-function data capture andcommunication device such as those described above may be remotelyguided in positioning the probe on a patient or other object by amore-skilled operator. Alternatively, or in addition, probe-positioningguidance may also be provided by an automated system.

In various embodiments, automatic guidance may be provided by anartificial intelligence system utilizing computer aided detectiontechniques to recognize features within an image and suggesting probemovements to more fully capture a desired target object. Alternatively,automatic guidance may be provided by obtaining optical images of theprobe as positioned on a patient (or other object). Such optical imagesmay be obtained with a web-cam or other computer-connected digitalcamera. Artificial Intelligence software configured to recognizeanatomical features may be used to guide the user to place the probe inan ideal position for imaging a target organ or other object. In someembodiments, a laser pointer or other indicator may be used to indicateprobe movement instructions to the operator.

Alternatively, the operator may be guided by a static or dynamic imageof a mannequin mock-up of a patient with illustrations of where to placethe probe in order to image a desired organ or other object.

Data Communications and Storage

In various embodiments, the systems of FIG. 5-FIG. 8 may be configuredto communicate information in addition to the raw echo data to anetwork-based storage device. For example, the systems may communicatevarious annotations and/or header information along with the raw echodata. Such annotation/header data may include information allowing aclinician or service provider to associate the raw echo data with apatient, such as an anonymized patient ID number. Additionally,information such as a date/time of the data capture session, a locationof the data capture session, imaging system settings used during imagecapture, a probe identifier, calibration data, environmental data, orother information that may be useful in beamforming or otherwise usingthe raw data.

Some examples of data reduction procedures are described above. Inaddition to those methods, additional processes may be performed topackage, compress, annotate or otherwise modify raw echo data prior totransmission to the data warehouse.

In some embodiments, each probe may be provided with its own hard-codedglobally unique identifier (GUID). The GUID, combined with the numericalrepresentation of the current date and time may be used as a basis forcreating a unique identifier for each data capture session. The uniqueidentifier may then be associated with the header data associated withand stored with the raw echo data from the data capture session. Theheader data may include elements such as: the date and time of the datacapture, the geographic location of the data capture, an ID numberidentifying the patient (or other object to be imaged), the settings ofthe probe, user interface settings used during live imaging, and anyother pertinent information. The unique identifier and headerinformation may be associated with the raw data set and stored locallyand/or in a data warehouse. The stored information may be referenced atany point in the future for an unlimited amount of accesses.

Although this invention has been disclosed in the context of certainpreferred embodiments and examples, it will be understood by thoseskilled in the art that the present invention extends beyond thespecifically disclosed embodiments to other alternative embodimentsand/or uses of the invention and obvious modifications and equivalentsthereof. Various modifications to the above embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, it is intended that the scope ofthe present invention herein disclosed should not be limited by theparticular disclosed embodiments described above, but should bedetermined only by a fair reading of the claims that follow.

In particular, materials and manufacturing techniques may be employed aswithin the level of those with skill in the relevant art. Furthermore,reference to a singular item, includes the possibility that there areplural of the same items present. More specifically, as used herein andin the appended claims, the singular forms “a,” “and,” “said,” and “the”include plural referents unless the context clearly dictates otherwise.Also as used herein, unless explicitly stated otherwise, the term “or”is inclusive of all presented alternatives, and means essentially thesame as the commonly used phrase “and/or.” It is further noted that theclaims may be drafted to exclude any optional element. As such, thisstatement is intended to serve as antecedent basis for use of suchexclusive terminology as “solely,” “only” and the like in connectionwith the recitation of claim elements, or use of a “negative”limitation. Unless defined otherwise herein, all technical andscientific terms used herein have the same meaning as commonlyunderstood by one of ordinary skill in the art to which this inventionbelongs.

What is claimed is:
 1. A method of ultrasound imaging comprising:transmitting an unfocused three-dimensional ping into an object from atransmitter element of a transducer array in a probe of a data capturedevice; receiving echoes of the unfocused three-dimensional ping with aplurality of receiver elements of the transducer array; convertinganalog signals from each of the plurality of receiver elements into afull dataset of digital sample sets, wherein the full dataset comprisesdigital sample sets from all the receiver elements; beamforming asub-set of the digital sample sets locally with the data capture deviceto form a limited quality video stream, wherein the sub-set comprisesfewer digital samples than the full dataset; displaying the limitedquality video stream on a first display coupled to the data capturedevice; communicating the full dataset to a network-based imagegeneration system; beamforming the full data set with the network-basedimage generation system to form a high quality video stream; anddisplaying the high quality video stream on a second display coupled tothe network-based image generation system.
 2. The method of claim 1,further comprising communicating the full dataset with a wired orwireless data transfer system.
 3. The method of claim 1, wherein thenetwork-based image generation system is selected from the groupconsisting of a tablet, a personal computer, a laptop, and a mobiledevice.
 4. The method of claim 3, wherein the limited quality videostream is displayed on the first display to a first user and the highquality video stream is displayed on the second display to a seconduser.